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ABSTRACT 


^  A  covariance  analysis  technique  using  the  Cramer-Rao  lower 
bound  for  assessing  filtering  and  prediction  performance  for  a 
class  of  nonlinear  systems  is  presented.  The  class  of  systems 
considered  is  nonlinear,  deterministic,  with  unknown  parameters. 
The  validity  of  this  technique  for  the  problems  considered  is 
justified  using  local  observability  theory  and  unbiased 
estimation  for  nonlinear  systems. 
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INTRODUCTION 


A  covariance  analysis  technique  by  means  of  the  Cramer-Rao 
lower  bound  for  assessing  filtering  and  prediction  performance 
for  a  class  of  nonlinear  systems  is  presented.  The  class  of 
systems  considered  is  nonlinear,  deterministic,  with  unknown 
parameters.  The  unknown  parameters  can  be  constants  or  are  known 
to  follow  nominal  time  functions  (parametric  or  numerical).  In 
the  latter  case,  the  proportional  constant  to  the  true  time 
function  is  the  unknown.  In  either  cases,  unknown  parameters  are 
to  be  jointly  estimated  with  the  state  variables. 

The  above  system  definition  fits  very  well  to  the  problem 
of  reentry  trajectory  estimation.  The  unknown  parameter  is  the 
ballistic  coefficient.  In  the  high  endoatmospheric  region,  the 
ballistic  coefficient  is  a  constant.  In  the  lower  altitude 
region,  the  ballistic  coefficient  is  a  time  (altitude  and  Mach 
number)  function.  In  certain  applications,  a  nominal  ballistic 
coefficient  profile  is  known  to  the  estimator. 

The  assumption  that  the  nonlinear  system  is  deterministic 
places  certain  restrictions  on  the  generality  of  the  ensueing 
analysis.  In  using  an  extended  Kalman  filter  for  state 
estimation  in  this  case,  a  process  noise  variance  is  selected  to 
represent  the  variability  of  the  unknown  parameter.  On  the  other 
hand,  if  the  maximum  likelihood  (batch)  estimator  is  used  with 
the  assumption  that  a  nominal  parameter  profile  is  available. 


'TV  ■-  u. 


then  the  underlying  system  can  be  modeled  as  deterministic.  For 
this  latter  situation,  we  therefore  feel  that  the  assumption  of  a 
deterministic  system  is  valid  for  many  applications. 

The  Cramer-Rao  bound  for  nonlinear  deterministic  systems 
has  been  shown  to  be  very  tight  for  the  trajectory  estimation 
problem  with  angle-only  measurements  [5].  This  is  the  basis  of 
the  analysis  technique  being  introduced  in  this  report.  The 
problem  of  trajectory  estimation  with  ballistic  coefficient  being 
the  unknown  parameter  provided  the  motivation  for  the  analysis 
method  described  herein. 

This  report  is  organized  as  follows.  The  problem 
considered  in  this  report  is  defined  using  system  and  measurement 
equations  in  Section  2.  The  Cramer-Rao  bound  theory  for 
deterministic  nonlinear  systems  is  reviewed  in  Section  3.  The 
validity  of  the  Cramer-Rao  bound  for  the  problems  considered  is 
justified  using  the  local  observibility  theory  and  the  unbiased 
estimation.  Covariance  equations  for  filtering  error  and 
prediction  errors  are  summarized  in  the  fourth  section. 
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2.  PROBLEM  DEFINITION 

Consider  the  following  continuous-time  system  with  discrete 
measurement  problem 

x  =  f  {x,  £)  (2.1) 

yk  =  h  (xk)  +  nk;  xk  =  x(tk)  (2.2) 

where  x  and  ^  are  state  and  measurement  vectors,  p  is  the  unknown 
parameter  vector  and  n^  is  the  measurement  noise  vector  which 
is  a  zero  mean,  white  Gaussian  sequence  with  covariance  Rfc. 

Two  cases  for  the  parameter  vector  p  are  considered. 

(1)  p  is  an  unknown  constant  vector. 

(2)  p  follows  a  vector  of  profiles  with  known  shape  but 
uncertain  in  absolute  value.  We  therefore  have 

pi(t)  =  ai  pi(t) 

where  pp(t)  is  a  time  function  (parametric  or  numerical)  known 
to  the  estimator  and  denotes  the  i-th  element  of  p°(t).  The 
corresponding  i-th  component  of  the  true  profile  is  denoted  by 
Pi(t)  and  the  proportional  constant  aj  becomes  the  unknown 
constant  to  be  estimated. 


In  either  cases  above,  we  augment  the  unknown  constants 
with  the  state  vector  x,  the  estimator  is  therefore  to  jointly 
estimate  x  and  the  unknown  constants.  Let  x  denote  either  the 
constant  vector  £  or  the  proportional  constant  vector  a,  we  then 
have 


X  ■  Cl 

and  the  augmented  state  vector  xa  is 
**  *  [xT,  ,TJ 


(2.3) 


(2.4) 
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3.  THE  CRAMER-RAO  BOUND  FOR  DETERMINISTIC  NONLINEAR  SYSTEMS 

3.1  Review  of  the  Cramer-Rao  Bound  Equations 

The  Cramer-Rao  bound  (CRB)  on  the  covariance  of  estimating 
x(tk)  based  upon  measurement  vectors  2^,  f°r 

unbiased  estimators  is  given  below. 


r<v  - 1  «i  HI  RI  Hi  °i 


(3.1) 


P(xk)  =  I(xk)_1  (3.1a 

where 

I(xir)  *  the  Fisher's  information  matrix  evaluated 
at  xk. 

P(xk)  *  the  Cramer-Rao  bound  evaluated  at  xk» 

Gi  *  ®ilGi+r*  h-2 , . . . ,  1 . 

Gk  ®  I  (an  identity  matrix). 

♦  ^  »  the  solution  of  3  ~~~  ~  Ft  t), 

*(t,  r)  *  I,  for  t  -  t^,  evaluated  at  t  =  t^+1 . 

Ffc  «  the  Jacobian  matrix  of  jf(x^(t)). 

»  the  Jacobian  matrix  of  h(x^). 


The  measurement  error  covariance  at  time  t* . 


The  derivation  of  the  above  results  may  be  found  in  [1], 

[2] .  Notice  that  the  PtXfc)  may  be  re-evaluated  at  any  time 
instance  and  this  is  accomplished  via  the  appropriate  choice  of 
the  composite  transition  matrix  G^. 

In  evaluating  the  performance  of  a  certain  nonlinear 
estimation  problem,  the  Cramer-Rao  bound  provides  several 
desirable  features.  It  is  easy  to  compute  when  compared  against 
other  bounds.  When  the  maximum  likelihood  estimator  (MLE)  can  be 
realized,  the  CRB  and  MLE  go  hand  in  hand.  It  is  well-known  that 
the  MLE  is  consistent,  asympotically . efficient,  and 
asymptotically  Gaussian,  [3] .  The  existence  of  MLE  therefore 
guarantees  that  the  CRB  is  at  least  asympotically  achievable. 

Clearly,  the  Cramer-Rao  Bound  (3.1a)  does  not  exist  if  the 
Fisher's  information  matrix  (3.1)  is  singular.  This  implies  that 
there  does  not  exist  any  unbiased  estimator  for  with  finite 
estimation  errors  (variances).  In  the  nonlinear  systems  theory, 

I ( xjt )  also  ties  with  the  nonlinear  observability  condition  and 
vhe  existence  of  a  particular  maximum  likelihood  estimation 
algorithm  as  we  shall  demonstrate  next. 


3.2  Local  Observability  Condition  and  MaziauB  Likelihood 
Estiaation 

In  [4] ,  the  use  of  Ilxfc)  in  examining  the  pointwise 
observability  at  jc*  was  presented.*  It  stated  that  if  Kx^) 
is  positive  definite,  then  the  system  is  observable  at  x*.  In 
here,  we  extend  the  above  condition  to  a  local  sphere  about 
Xfc.  We  first  present  the  following  theorem. 

Theorem  3 . 1 

Given  a  positive  definite  symmetric  matrix  A  with 
eigenvalues  Xn  >  Xn_i  >_•••>_  Xi>0,  if  B  is  symmetric  and 
| | A-B | | <  X i ,  then  B  is  positive  definite. 

Proof ; **  Using  the  definition 

X,  .  <  ax,  X  > 

and  A-B  being  symmetric,  we  have 

I  I A-B I  I  =  max  <  (A-B)  x,  x  > 

II  H  ||x||-, 

Thus,  for  any  unit  vector  x» 

*  The  observability  Grammian  of  [4]  is  the  same  as  Hx^) 
with  Ri  being  set  to  an  identity  matrix. 

**  Proof  of  the  theorem  is  due  to  Dr.  R.  B.  Holmes. 
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<  (A-B)x,  x  >  =  <  Ajc,  x  >  -  <  Bx ,  x  > 

<  *1 

<  <  Ax,  x  >  , 

Clearly,  <Bx,  x>  >  0,  then  B  is  positive  definite.  QED. 

We  can  apply  the  above  theorem  to  define  a  sphere  around  a 
point  x  where  I(x)  is  positive  definite  such  that  all  points 
within  this  sphere  will  have  observability  Grammian  being 
positive  definite. 

Theorem  3.2 

If  I(^)  is  positive  definite  at  ^0,  then  one  can 
construct  a  sphere  S  *  :  ||z~Zo||  <  r}  such  that  for  all  ^ 

in  S,  I(^)  is  positive  definite  and  the  radius  of  S  can  be 
choosen  as 

r  ■  min  {r^,  rz] 

where  |  |x-^o|  |  <  V  “  I<XQ>  1 1  <  L  ||z-*o||'  and  r2=x1/L* 

Proof  Given 

ll*-*oll  <  rt 

||K2>  -  Kio>||  <  «•  |  Ix-Xo!  |  <  1  •  r, 

Consider  two  cases: 

(a)  Assuming  r1  <  rjr  or  L»r^  <  X^,  then 
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this  gives  i(^)  >  0. 

(b)  Assuming  r2^ri,  or  Xj^L^r^,  then  choose  a  new 
such  that 

II*'  -ioll  <  X,/L 

Then  ||l<]f)-I<lo)||  <  L.  ||2'-J^||  -  X, 

this  gives  I(^')  >  0.  QED. 

The  above  theorem  presents  the  fact  that  when  I(xfc)  is 
positive  definite,  then  the  reconstruction  of  x*  at  a  local 
region  can  be  made  and  the  size  of  this  region  (the  sphere  S)  can 
be  estimated,  we  next  illustrate  a  particular  realization  of  the 
Maximum  Likelihood  estimator  where  the  existence  of  this 
algorithm  also  depends  on  the  invertibility  of  Itxjc). 

An  iterative  algorithm  in  implementing  the  Maximum 
Likelihood  estimator  for  estimating  the  state  vector  with  angle 
only  measurements  was  presented  in  [5]  .  Let  x£  denote  the 
nth  iteration  on  estimating  x^,  then  the  following  algorithm, 
derived  in  [5],  gives  the  n+lst  iterative  solution. 


(3.2) 


where  the  covariance  matrix  P(x£)  takes  the  same  functional 
form  as  the  Cramer-Rao  Bound  defined  in  (3.1).  Clearly,  the 
existence  of  P(Xkn)  is  dependent  upon  the  invertibility  of 
I(jycn)»  It  was  shown  in  [5]  that  the  above  algorithm 
provides  estimates  which  asymptotically  approach  the  Cramer-Rao 
Bound.  This  was  not  surprising  due  to  the  theoretically 
justified  property  of  the  Maximum  Likelihood  estimator. 

The  derivation  of  (3.2)  was  based  upon  minimizing  a 
quadratic  error  criterion,  i.e.. 


min  J  -  l  (^-hfx,))  TRi~lx.i-h(*i ) ) 
xk  i«1  1  “  -1  i  i-i 


(3.3) 


In  the  case  of  designing  a  nonlinear  observer,  the  Bq.  (3.2)  can 
be  used  simply  be  setting  R*  to  an  identity  matrix.  The 
convergence  of  (3.2)  is  guaranteed  if  the  J  above  is  convex  in  a 
local  region  about  x*  and  the  initial  guess  x^0  is 
contained  in  this  region. 


3.3  Su—ary 

In  this  section,  we  have  presented  the  fundamental 
equations  on  the  Cramer-Rao  bound  for  systems  defined  with  (2.1) 
and  (2.2).  We  have  also  tied  the  relationship  between  the 
invertibility  of  the  Fisher's  information  matrix  to  the  local 
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observability  of  nonlinear  systems  and  the  existence  of  a 
particular  Maximum  Likelihood  algorithm,  we  summarize  our 
findings  below. 

(1)  The  computation  of  the  Cramer-Rao  bound  is  possible 
only  if  the  Fisher's  information  matrix  is  positive  definite. 

(2)  When  this  is  the  case,  the  nonlinear  system  is  locally 
observable. 

(3)  A  maximum  likelihood  estimation  algorithm,  which  also 
requires  the  Fisher's  information  matrix  to  be  nonsingular,  can 
be  constructed.  In  the  noise  free  measurement  case,  this 
algorithm  becomes  an  observer  and  the  convergence  is  guaranteed 
if  the  quadratic  error  criterion  is  convex  about  x*  and  the 
initial  guess  jc£  is  contained  in  this  region. 


4.  ERROR  COVARIANCE  EQUATIONS 

With  the  use  of  the  Cramer-Rao  Bound  for  estimation 
performance  evaluation  justified,  we  now  present  computational 
algorithms  for  filtering  and  prediction  errors. 

4 . I  Pilter ing  Errors 

The  computation  of  (3.1)  involves  matrix  inversion.  When 
sufficient  number  of  measurements  have  been  collected  so  that  the 
information  matrix  Kxjt)  becomes  nonsingular,  then  Eq.  (3.1) 
can  be  replaced  by  the  familiar  matrix  Riccati  equation. 

Examining  (3.1),  one  can  write 

T  T  —1 

K2W  -  1I)  H?  V  H4  Gt 

’  Rklt  “k+1  +  *X  ^k'  <4-" 

Let 

-  *k  p<ik»  *k  <4-2> 

One  can  obtain  the  following  result  with  direct  application  of 
the  Matrix  Inversion  Lemma  to  Eq.  (4.1). 

P(-k+1)  "  Pk+1  [I~  Hk+1 (Hk+1Pk+1Hk+1+  )  ^k+l^k+l1  (4,3) 

The  computation  of  P^+i ,  Eq.  (4.2),  can  also  be  replaced  by 
solving  for  Pfc+i  using  the  following  matrix  differential 
equation. 


with  initial  condition  Pfxfc)  at  t  ■  tfc. 

Equations  (4.3)  and  (4.4)  give  the  filtering  error.  They 
are  applied  as  soon  as  I(xjc)  becomes  nonsingular. 

4.2  Prediction  Errors 

The  prediction  error  equation  is  (4.4)  (or  (4.2)),  by 
solving  it  at  the  time  desired  with  the  initial  condition  set  at 
the  last  point  of  measurements. 

In  the  problem  of  trajectory  estimation  with  the  ballistic 
coefficient  as  the  unknown  parameter,  two  situations  may  occur 
for  the  problem  of  trajectory  prediction.  This  is  due  to  the 
fact  that  the  variations  of  ballistic  coefficients  (i.e., 
deviations  from  a  nominal  time  profile)  are  usually  known  to 
within  a  priori  bounds.  When  the  ballistic  coefficient 
estimation  (filtering)  error  is  below  those  errors  characterized 
by  a  priori  bounds,  the  trajectory  prediction  error  is  obtained 
by  solving  Eq.  (4.4)  (or  (4.2)).  On  the  other  hand  when  such  is 
not  the  case,  the  trajectory  prediction  error  should  only  be 
limited  to  errors  induced  by  the  a  priori  bounds.  In  this 
latter  case,  the  error  equation  can  be  derived  using  the 
following  linear  equation  approximation.  Let  the  linear  system 


be  modeled  as 


x  =  P  x  +  G  y  (4.5) 

Notice  that  the  unknown  parameter  is  treated  as  a  driving  force 
term.  The  solution  of  (4.5)  can  be  obtained  via  the  transition 
matrix  «(t,  t)  as 

t 

xt  =  *(t'  to)  2o  +  [  *(t'  T)  G  i(T)  dT  (4*6) 

o 

where  xQ  is  the  initial  condition.  Let  x't  denoted  a 

perturbed  solution  due  to  perturbation  6 x  in  the  parameter  vector 

x,  one  obtains 

t 

xt  *  *  (t,  tQ)  x_  +  /  *(t,  t)  G(fi+fix)  dt  (4.7) 

to 

Let  l  denote  the  outer  product  of  the  trajectory  perturbation 
6xt*  it  can  be  shown  that 

t 

fix,.  *  /  *(t,  t)  G5y  dr  (4.8) 

fco 

and 

l  ■  (4.9) 

The  above  analysis  suggests  the  following  two  separate 
procedures  for  trajectory  prediction  when  a  priori  bounds  on  the 
unknown  parameter  are  available. 
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(1)  When  the  parameter  estimation  error  is  smaller  than 
the  a  priori  bound,  use  the  covariance  differential  equation 
(Eq.  (4.2)  or  (4.4))  to  solve  for  trajectory  prediction  error. 

(2)  When  the  parameter  estimation  error  is  large, 
calculate  estimation  error  assuming  perfect  knowledge  on  the 
parameter  value,  then  obtain  trajectory  prediction  error  using 
trajectory  perturbation  Eqs.  (4.8)  -  (4.9). 
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SUMMARY 


In  this  report,  we  have  presented  equations  for  calculating 
filtering  and  prediction  errors  for  systems  with  uncertain 
parameters.  The  results  are  based  upon  the  Cramer-Rao  bound  on 
covariance  in  state  estimation.  This  analysis  is  motivated  by 
the  problem  of  trajectory  estimation  with  uncertain  parameters. 

The  major  findings  of  this  report  are  summarized  below. 

( 1 )  unknown  constant  parameters  are  modeled  as  constant 
state  variables. 

(2)  Unknown  time-varying  parameters  with  known  time 
profiles  require  the  modeling  of  proportional  constants  as 
constant  state  'variables. 

(3)  The  use  of  the  Cramer-Rao  Bound  as  an  analysis  tool  is 
discussed  and  its  relationship  with  the  nonlinear  observability 
condition  and  the  existence  of  a  maximum  likelihood  algorithm  is 
explored . 

(4)  Filtering  errors  are  obtained  by  solving  the 
Cramer-Rao  bound  equation. 

(5)  Prediction  errors  due  to  filtering  error  alone  or  with 
the  knowledge  of  parameter  a  priori  bounds  are  also  obtained. 

Application  of  the  analysis  discussed  herein  to  the 
trajectory  estimation  problem  will  be  published  in  a  future 
report. 
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